Deeply-Fused Nets

نویسندگان

  • Jingdong Wang
  • Zhen Wei
  • Ting Zhang
  • Wenjun Zeng
چکیده

In this paper, we present a novel deep learning approach, deeply-fused nets. The central idea of our approach is deep fusion, i.e., combine the intermediate representations of base networks, where the fused output serves as the input of the remaining part of each base network, and perform such combinations deeply over several intermediate representations. The resulting deeply fused net enjoys several benefits. First, it is able to learn multi-scale representations as it enjoys the benefits of more base networks, which could form the same fused network, other than the initial group of base networks. Second, in our suggested fused net formed by one deep and one shallow base networks, the flows of the information from the earlier intermediate layer of the deep base network to the output and from the input to the later intermediate layer of the deep base network are both improved. Last, the deep and shallow base networks are jointly learnt and can benefit from each other. More interestingly, the essential depth of a fused net composed from a deep base network and a shallow base network is reduced because the fused net could be composed from a less deep base network, and thus training the fused net is less difficult than training the initial deep base network. Empirical results demonstrate that our approach achieves superior performance over two closely-related methods, ResNet and Highway, and competitive performance compared to the state-of-the-arts.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

On the Connection of Deep Fusion to Ensembling

In this paper, we provide a systematic study to the prevailing ResNet architecture by showing a connection from a general deeply-fused net view to ensembling. We start by empirically demonstrating the resemblance between the expanded form of the deeply-fused net and an ensemble of neural networks. Our empirical results uncover that the deepest network among the ensemble components does not cont...

متن کامل

Non Sequential Semantics for Contextual P/T Nets

The problem of nding a true concurrent semantics for con-textual P/T nets has not been deeply studied yet. The interest for such a semantics has been renewed by some recent proposals to equip mobile process algebras with a net semantics based on this model. In 3] we proposed a causal semantics for such nets, in this paper we study the non sequential semantics of contextual P/T nets. The semanti...

متن کامل

A Causal Semantics for Contextual P/T Nets

The problem of nding a true concurrent semantics for contextual P/T nets has not been deeply studied yet. The interest for such a semantics has been renewed by some recent proposals to equip mobile process algebras with a net semantics based on this model. In this paper we move a step towards the deenition of such a semantics: we rst propose a causal semantics for P/T nets and we prove that thi...

متن کامل

Hydrogen bonding in substituted nitroanilines: isolated nets in 1,3-diamino-4-nitrobenzene and continuously interwoven nets in 3,5-dinitroaniline.

Molecules of 1,3-diamino-4-nitrobenzene, C(6)H(7)N(3)O(2), are linked by N-H.O hydrogen bonds [N.O 2.964 (2) and 3.021 (2) A; N-H.O 155 and 149 degrees] into (4,4) nets. In 3,5-dinitroaniline, C(6)H(5)N(3)O(4), where Z' = 2, the molecules are linked by three N-H.O hydrogen bonds [N.O 3.344 (2)-3.433 (2) A and N-H.O 150-167 degrees] into deeply puckered nets, each of which is interwoven with its...

متن کامل

Througput Lower Bounds for Markovian Petri Nets: Transformation Techniques

This paper addresses the computation of lower bounds for the steady-state throughput of stochas-tic Petri nets with immediate and exponentially distributed service times of transitions. We try to deeply bridge stochastic Petri net theory to untimed Petri net and queueing networks theories. Previous results for general service time distributions are improved for the case of Markovian nets by con...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • CoRR

دوره abs/1605.07716  شماره 

صفحات  -

تاریخ انتشار 2016